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1.  Background 


Visualizing  future  battlespaces  (including  decidedly  nonconventional  ones)  is  an  important  goal 
for  the  Objective  Force  and  Future  Combat  Systems  (FCS).  Technological  advances  will  result  in  a 
knowledge-rich  battlefield  with  virtual  planning,  multi-modal  visualizations,  disbursed  operations, 
and  highly  intelligent  automated  systems.  Too  often,  however,  technological  advances  are  de¬ 
signed  without  enough  consideration  for  the  warfighters  who  use  them  (Barnes,  1997).  Obviously, 
the  effects  of  visualization  will  cascade  over  the  entire  battlefield  and  will  influence  the  common 
and  specialized  views  of  the  battle  at  all  echelons.  Yet  the  demand  for  improved  technology 
outpaces  understanding  of  the  benefits  of  the  various  techniques.  The  purpose  of  this  report  is  to 
develop  an  understanding  of  the  theoretical  and  empirical  underpinnings  of  visualization  research 
in  order  to  create  design  principles  for  future  military  visualization  systems.  Visualization  is  the 
process  that  commanders  use  to  envision  new  tactical  alternatives.  It  is  a  bridge  between  human 
knowledge  and  “seeing”  new  solutions.  However,  the  ability  to  visualize  is  based  on  heuristic 
processes  that  have  cognitive  costs  as  well  as  benefits.  Effective  synergy  between  humans  and 
visualization  systems  must  be  based  on  design  principles  that  engender  human  strengths  and 
limitations. 

The  U.S.  Army  definition  of  battlespace  visualization  implies  not  only  “imaging”  the  battlespace 
but  also  more  significantly  generating  objectives  and  possible  tactics  to  achieve  these  objectives: 

The  process  whereby  die  commander  develops  a  clear  understanding  of  his 
current  state  with  relation  to  the  enemy  and  environment,  envisions  a  desired  end 
state  which  represents  mission  accomplishments,  and  then  subsequendy  visualizes 
the  sequence  of  events  that  will  move  his  forces  from  the  current  state  to  the  end 
state  (Department  of  the  Army,  1997). 

Thus,  it  is  important  not  to  confuse  visualization  with  the  depiction  of  batdespace  information. 
Understanding  the  topographical,  spatial,  and  force  dispositions  on  the  batdefield  is  a  necessary 
component  of  visualization  but  is  not  sufficient.  The  ultimate  purpose  of  visualization  aids  is  to 
increase  the  commander’s  ability  to  understand  the  batde  dynamics,  consider  options,  and  predict 
outcomes.  For  that  reason,  I  have  included  a  number  of  topics  related  to  decision  making  in  future 
battlespaces,  including  visualization  issues  raised  by  uncertainty  representation,  naturalistic 
decision  making,  automated  systems,  and  complexity  theory.  The  purpose  of  this  report  is  to 
identify  important  visualization  issues  that  impact  the  commander’s  ability  to  understand  and 
predict  the  combat  situation.  The  discussion  focuses  on  design  and  research  issues  likely  to  impact 
FCS. 
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2.  Overview  and  General  Model  of  Visualization 


Figure  1  portrays  the  visualization  process  from  the  beginning  of  the  planning  procedures  to 
understanding  the  current  state  to  mental  projections  of  possible  future  states.  Visualization  is  not 
concerned  solely  with  the  perception  of  the  current  combat  environment.  It  is  both  historical  and 
teleological;  commanders  are  continuously  planning  and  executing,  based  on  past  trends  and 
their  relation  to  the  desired  “end  state.”  Perception  is  an  important  component  of  visualization, 
but  in  reality,  visualization  is  more  concerned  with  problem  solving  than  with  immediate 
awareness.  However,  the  type  of  decision  space  in  which  the  commander  must  operate  is  not 
straightforward.  To  illustrate  the  type  of  decision  processes  that  the  commander  uses  to 
“visualize  the  sequence  of  events  that  will  move  his  forces  from  the  current  state  to  the  end 
state,”  three  basic  decision  types  are  shown  as  solid,  short  dashes,  or  long  dashed  lines.  The 
expert  commander  quite  often  is  able  to  visualize  the  battlespace  in  terms  of  a  single  preferred 
solution  (Klein,  1999),  and  this  type  of  visualization  process  is  shown  as  the  solid  line 
representing  an  almost  deterministic  process. 


r  Unexpected  Event 
Perceived  Option  1 

►  Current  Situation  j^^^^^Favored  Option 
Emerging  Trends  Perceived  OptionN 

Unexpected  Event 


Figure  1.  Deterministic,  probabilistic,  and  complex  decision  processes  for  visualization  of 
future  events. 


However,  many  decision  processes  and  much  of  the  traditional  decision  research  assumes  that 
the  decision  maker  considers  probabilities  associated  with  multiple  options  (shown  as  dashed 
lines).  For  this  class  of  decisions,  the  commander  must  consider  a  set  of  options,  uncertainties, 
and  consequences,  all  subject  to  cognitive  biases  that  can  distort  the  visualization  process 
(Wickens  &  Hollands,  2000).  Finally,  decisions  must  be  made  in  which  there  are  neither  single 
solutions  nor  probabilistic  ones.  These  decisions  are  those  that  must  be  made  in  complex  or 
chaotic  decision  spaces.  The  long  dashed  line  is  meant  to  convey  the  notion  that  not  all  the 
options  are  known  or  the  decision  space  is  so  volatile  that  a  stable  probabilistic  structure  does  not 
exist  (Waldrop,  1992).  In  truth,  actual  combat  decisions  may  not  fit  neatly  into  any  of  these 
categories.  However,  the  categories  subsume  important  research  topics  in  decision  making  and 
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the  distinctions  are  useful  in  the  delineation  of  many  of  the  cognitive  issues  that  impact  human 
visualization  processes. 

Two  basic  processes  underlie  visualization:  current  situational  understanding  and  prediction. 
Situational  understanding  entails  the  processes  used  to  visualize  (and  understand)  the  current 
situation.  The  visualization  technologies  to  support  situational  understanding  in  a  military 
environment  are  fairly  well  developed;  unfortunately,  the  human  performance  implications  are 
not  as  well  defined.  The  focus  in  these  sections  is  on  the  human  and  operational  design  issues  for 
the  various  technologies  including  terrain  visualization  systems,  military  symbology,  knowledge 
walls,  and  some  of  the  more  recent  cognitive  concepts  for  abstract  visualizations.  Although  the 
emphasis  is  on  military  related  research,  basic  cognitive  and  human  processing  research  is 
discussed  whenever  appropriate.  The  second  major  division  is  prediction.  Prediction  includes 
those  processes  that  permit  the  warfighter  to  “see”  possible  solutions  in  order  to  achieve  end 
states.  Discussion  entails  design  issues  related  to  the  decision  processes  outlined  in  Figure  1.  In 
particular,  military  research  based  on  traditional  heuristic  models  of  probabilistic  decision 
making  is  contrasted  to  results  based  on  naturalistic  decision-making  (NDM)  models.  I  conclude 
that  the  two  methods  represent  complementary  approaches  to  understanding  the  role  of  visuali¬ 
zation  in  prediction:  NDM  research  indicates  the  efficiencies  of  expert  decision  making,  where¬ 
as  the  heuristic  research  indicates  the  other  side  of  the  coin:  cognitive  biases.  Visualization 
processes  related  to  automation  are  discussed  as  a  binary  decision  involving  biases  toward 
manual  and  automated  choices,  depending  on  the  quality,  presentation  mode,  and  type  of  infor¬ 
mation  presented.  Finally,  complex  military  environments  are  discussed  in  terms  of  various 
decision  aiding  and  visualization  techniques  that  are  being  developed  to  reduce  the  complexity  in 
these  environments.  Again,  the  purpose  of  the  report  is  to  help  designers  and  researchers  under¬ 
stand  the  human  issues  related  to  visualizing  future  battlespaces.  Design  principles  are  suggested 
when  the  data  so  warrant;  in  other  cases,  unresolved  research  issues  are  discussed  in  their  stead. 
Also,  visualization  is  a  large  topic  area;  by  necessity,  this  report  covers  only  a  limited  portion. 


3.  Current  Situational  Understanding 


3.1  Terrain 

The  history  of  warfare  is  replete  with  examples  of  the  use  of  successful  tactics  that  depend  on  a  - 
thorough  understanding  of  the  local  terrain.  From  Vicksburg  to  the  Plains  of  Abraham  to  the 
African  deserts,  victory  depended  on  the  use  of  terrain  characteristics  as  part  of  the  battle  plan. 
Modem  display  technology  allows  the  operator  to  see  realistic  views  of  the  battlespace  that  use 
multiple  angles,  altitudes,  and  dimensional  options.  The  visualization  issue  is  to  determine  how 
well  combatants  perform  military  tasks  when  they  use  different  terrain-viewing  technologies. 
Dimensionality  has  been  researched  thoroughly  in  the  aviation  domain.  Most  of  the  research 
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involves  a  comparison  of  two-dimensional  (2-D)  planar  views  with  three-dimensional  (3-D) 
rendered  views  while  slant  range  and  angle  are  varied.  The  general  finding  is  that  there  are  as 
many  tasks  in  which  3-D  displays  impede  performance  as  those  for  which  they  enhance  per¬ 
formance  (Banks  &  Wickens,  1999).  This  is  not  surprising;  projecting  a  3-D  representation  onto 
a  2-D  surface  results  in  space  distortions,  ambiguous  locations  along  the  line  of  sight  (LOS),  and 
foreshortening  phenomena  (St.  John,  Cowen,  Smallman,  &  Oonk,  2001).  For  example,  Banks 
and  Wickens  reported  that  in  judging  the  relative  positions  of  aircraft,  most  studies  showed 
superior  performance  for  2-D  formats,  which  indicated  difficulties  in  keeping  track  of  both  the 
vertical  and  horizontal  information  for  the  3-D  displays.  In  contrast,  they  found  that  when  the 
task  involved  terrain  understanding,  3-D  resulted  in  better  performance.  Naval  researchers  also 
found  many  useful  applications  for  3-D  renderings  of  littoral  operations  such  as  mine  warfare, 
command  and  control  and  air  operations  (Eddy  &  Kribs,  1999). 

Specific  Army-related  experiments  involved  map  tasks  that  used  experienced  cadres  from  the 
United  States  Military  Academy.  In  a  series  of  experiments,  the  cadres  performed  various  tac¬ 
tical  map  tasks  using  3-D  oblique,  immersed  3-D,  or  2-D  planar  terrain  views  rendered  on  a 
silicon  graphics  system.  For  simple  map  tasks,  distance  judgments  were  superior  with  2-D  views, 
mobility  judgments  were  unaffected  by  view  type,  and  LOS  judgments  showed  increased  ac¬ 
curacy  with  an  immersed  3-D  view  of  the  mountainous  terrain.  For  tasks  that  required  tactical 
decisions,  immersion  in  the  terrain  appeared  to  cause  cognitive  tunneling  among  the  surrogate 
tank  commanders.  The  participants  ignored  important  information  that  was  not  in  the  frontal 
field  of  view,  even  when  this  information  was  available  on  an  insert;  this  was  partly  attributable 
to  the  difficulty  of  switching  attention  between  two  displays  (Wickens,  Thomas,  &  Young,  2000; 
Thomas  &  Wickens,  1999).  In  general,  being  immersed  in  the  problem  space  has  cognitive  costs 
as  well  as  benefits;  it  is  easy  for  the  operator  to  narrow  his  or  her  attentional  focus  and  ignore 
important  peripheral  information.  Other  research  has  generalized  these  findings  to  more  abstract 
3-D  domains  which  indicate  that  being  immersed  within  the  problem  space  helps  one  in  naviga¬ 
ting  through  and  remembering  local  features;  however,  if  the  task  is  to  comprehend  global 
features  of  this  space,  then  “god’s  eye”  exocentric  views  (views  from  outside  the  problem  space) 
proved  superior  (McCormick,  Wickens,  Banks,  &  Yeh,  1998). 

U.S.  Navy  researchers  investigated  the  dimensionality  issue  (2-D  versus  3-D)  as  well  and  found 
very  similar  trends  using  terrain-related  and  abstract  tasks.  They  concluded  that  2-D  displays 
were  superior  for  understanding  relative  positions  in  space  but  that  3-D  rendered  views  aided  in 
tasks  that  required  understanding  terrain  contour  and  depth  cues  (e.g.,  LOS  judgments)  (St.  John 
et  al.,  2001).  In  summary,  the  research  indicates  that  no  one  view  of  the  terrain  is  superior  in  all 
situations.  For  force  disposition  and  an  overall  tactical  understanding,  2-D  planar  views  have  the 
advantage  of  simplicity  and  veridical  scale.  On  the  other  hand,  an  understanding  of  the  terrain 
itself,  its  general  topography,  and  possible  high  and  low  spots  is  enhanced  by  3-D  views.  Situa¬ 
tions  that  require  mobility  through  the  terrain  (such  as  rehearsal  during  the  planning  process) 


4 


benefit  from  3-D  immersed  displays  with  the  caveat  of  the  lowering  of  the  attentional  bandwidth 
of  the  observer  (Wickens  &  Hollands,  2000). 

The  problem  is  more  complicated  when  3-D  displays  are  used  for  portraying  dimensions  other 
than  spatial  ones.  For  example,  Hollands,  Pierce,  and  Magee  (1998)  investigated  the  ability  of 
observers  to  track  vehicular  movements  on  the  X- Y  plane  with  time  forming  the  third  potential 
dimension.  Judgments  about  the  convergence  or  divergence  of  and  distance  between  the  two 
vehicles  were  generally  more  accurate  for  2-D  versus  3-D  presentations,  which  again  suggests 
that  costs  are  associated  with  processing  a  third  dimension.  In  an  ensuing  study,  mesh  and  line 
enhancements  were  added  to  the  3-D  representation;  the  results  indicated  a  clear  benefit  for  the 
enhanced  3-D  conditions  in  contrast  to  the  3-D  control  and  the  2-D  conditions  (Meserth  & 
Hollands,  1999).  What  was  disquieting  about  the  latter  study  was  the  evidence  of  optical  illu¬ 
sions  related  to  curvature,  which  caused  the  observer  to  be  biased  toward  convergence  judgments 
when  the  vehicles  were  close  together.  Interestingly,  the  type  of  illusion  seemed  to  interact  with 
the  type  of  enhancement.  Problems  with  linear  perspective  apparently  caused  a  divergence  bias 
for  vehicular  tracks  that  were  widely  separated  for  conditions  where  a  line  was  inserted  between 
the  tracks  as  a  visual  cue.  This  is  important  because  the  original  intent  of  the  research  was  to 
show  that  emergent  cues  aided  in  trend  judgments;  the  experiments  also  suggest  that  emergent 
cues  can  display  illusionary  information. 

The  point  to  all  this  is  that  multiple  viewing  capabilities  will  be  required  for  any  sufficiently  com¬ 
plex  battlespace  to  respond  to  changing  task  requirements  and  to  monitor  various  battlespace 
slices  simultaneously  (Barnes  &  Wickens,  1998).  The  commander  will  need  to  switch  and  com¬ 
pare  battle  views  much  like  the  television  networks  do  in  the  so-called  Monday  night  football 
paradigm  (the  television  camera  operator  chooses  the  best  view  from  multiple  camera  shots).  The 
narrative  of  the  battle  situation  must  be  maintained  while  views  are  switched — something  that  has 
proved  to  be  a  significant  human  performance  issue.  Attempting  to  maintain  the  correspondence 
between  two  views  is  difficult;  doing  the  same  thing  among  multiple  views  will  tax  the  observer 
to  an  even  greater  extent.  Somehow,  a  cognitive  narrative  needs  to  be  woven  so  that  the  observer 
sees  the  different  views  as  components  of  a  perceptual  whole  (cf,  Woods’  notion  of  visual 
momentum;  Woods,  1984).  For  military  tasks  that  required  combining  and  switching  between 
2-D  and  3-D  views,  the  Navy-sponsored  researchers  investigated  five  view  correspondence 
strategies  and  found  that  two  of  the  approaches  were  the  most  promising.  Interestingly,  the 
obvious  solution  of  a  side-by-side  view  was  not  effective  because  of  the  view-switching  problem 
alluded  to  previously  (see  Figure  2).  Based  on  operator  performance  for  both  the  antenna-placing 
task  and  the  terrain  correspondence  test,  the  two  superior  options  were  the  time  “morph”  (i.e., 
gradually  shifting  software  views)  and  the  overlay  displays.  The  time  morph  allowed  the  observer 
to  replace  a  2-D  topographic  map  with  a  3-D  representation  for  a  2-second  duration  by  holding 
down  the  space  bar.  The  overlay  option  combined  the  2-D  topographic  and  the  3-D  renderings  on 
the  same  display.  More  research  needs  to  be  done  to  establish  general  principles  and  methods  for 
view  switching,  but  the  initial  research  indicates  that  a  number  of  clever  software  techniques  may 
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help  humans  maintain  correspondence  among  multiple  views  of  the  battlespace  (St.  John, 
Smallman,  &  Cowen,  2002). 


Figure  2.  Views  (2-D  and  3-D)  of  the  same  terrain  for  the  integrated  display  experiments  (St.  John  et  al., 
2002). 


This  research  pertained  mainly  to  3-D  rendering  on  a  flat  surface.  There  are,  however,  two  addi¬ 
tional  techniques  that  produce  3-D  visualization  effects:  motion-induced  cues  and  stereopsis 
(Kaiser  &  Profitt,  1992).  At  this  time,  few  3-D  military  displays  use  stereopsis  or  motion  as 
techniques  for  producing  3-D;  this  should  change  as  virtual  technology  becomes  more  accessible 
and  the  equipment  required  to  obtain  these  effects  becomes  more  cost  efficient.  Researchers 
investigating  the  different  types  of  3-D  views  found  that  combining  induced  motion  and  stereo¬ 
scopic  cues  with  monocular  cues  strengthened  the  perceived  3-D  effects  in  a  linear  fashion,  that 
is,  each  cue  improved  depth  perception  in  an  additive  fashion  (Sollenberger  &  Milgram,  1993). 
Although  these  cues  show  performance  gains  for  certain  tasks,  they  also  have  their  drawbacks, 
including  the  obvious  ones  such  as  the  specialized  headgear  necessary  for  some  3-D  applications 
(Kaiser  &  Profitt,  1992;  Wickens  &  Hollands,  2000;  Yeh  &  Silverstein,  1992).  The  benefits  of 
creating  a  virtual  3-D  world  with  motion  and  stereoscopic  cues  need  to  be  carefully  assessed 
against  possible  costs.  Costs  associated  with  3-D  in  general  and  alarming  reports  of  simulation 
sickness  in  particular  need  to  be  investigated  more  thoroughly  before  these  technologies  are 
widely  adopted  in  military  environments.  The  relationship  of  these  technologies  to  human 
performance  is  just  beginning  to  be  understood.  Wickens  and  Hollands  outline  a  variety  of 
performance  costs  for  virtual  environments  (VE):  serious  time  lags  between  the  observer’s 
actions  and  VE,  perceptual  illusions,  observer  disorientation,  and  simulation  sickness.  As  the 
technology  improves,  some  of  these  costs  will  be  ameliorated,  but  the  prevalence  of  motion 
sickness  symptoms  with  more  conventional  flight  simulators  (with  only  visual  cues)  should 
remind  the  designer  that  3-D  VE  might  cause  serious  disruptions  if  the  human-related  issues  are 
not  evaluated  thoroughly. 
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3.2  Displaying  Tactical  Information 

The  military  situation  map  and  the  use  of  standard  symbols  have  been  part  of  U.S.  military  tradi¬ 
tion  at  least  since  the  Civil  War.  These  representations  have  allowed  commanders  and  staff  the 
ability  to  plan,  discuss,  and  even  rehearse  the  coming  battle  with  virtually  no  technological 
overhead.  The  present  discussion  focuses  on  the  perceptual  and  cognitive  requisites  for  using 
standard  military  symbols  as  a  visualization  tool.  Searching  for  target  symbols  is  among  the  most 
primitive  tasks  that  the  commander  performs  in  order  to  use  the  situation  map,  and  yet,  human 
performance  of  this  seemingly  simple  task  is  fairly  complex.  In  general,  search  time  and  accu¬ 
racy  degrade  as  a  function  of  the  total  amount  of  information  on  the  situation  display  (Teichner 
&  Krebs,  1974;  Teichner  &  Mochamuk,  1979;  Wickens  &  Hollands,  2000).  However,  at  the 
same  time,  humans  have  developed  a  number  of  specialized  mechanisms  that  allow  them  to 
process  large  amounts  of  information  efficiently.  The  best  understood  of  these  skills  is  pre- 
attentive  processing  which  early  investigators  noted  in  both  the  visual  and  auditory  domains  (cf, 
Neisser,  1967).  Basically,  humans  scan  their  environment  to  note  important  target  items  (usually 
something  out  of  place)  without  focusing  attention  on  any  particular  item.  This  mechanism 
allows  humans  to  process  complex  scenes  while  focusing  on  important  change  cues.  The  same 
processes  can  be  used  to  search  for  target  items  in  a  complex  display  as  long  as  the  target  item 
contains  features  that  contrast  with  the  non-target  items.  Pre-attentive  processing  (defined  as 
search  time  being  unrelated  to  the  number  of  non-targets  in  the  display)  is  particularly  efficient 
because  we  can  detect  the  target  feature  in  the  general  clutter  with  minimal  interference  from 
non-targets.  There  are  even  cases  when  the  target  symbol  is  searched  for  more  rapidly  as  the 
number  of  non-targets  increases  (Bacon  &  Egeth,  1991)  (presumably  because  the  non-targets 
create  a  contrasting  background  from  which  the  target  feature  “pops  out”). 

However,  the  number  of  symbol  sets  in  which  this  is  possible  is  limited.  For  most  symbol 
searches,  humans  use  focused  attention,  which  is  more  time  consuming  and  less  efficient. 
Military  symbols  that  are  similar  to  other  symbols  or  those  for  which  the  warfighter  must  tally 
more  than  a  single  attribute  (e.g.,  unit  type  and  status)  must  be  searched  for  with  the  slower  but 
generally  more  accurate  focused  search  strategy.  Apparently,  this  is  a  not  an  all-or-nothing 
phenomenon  because  certain  target  cues  (such  as  color  and  size  contrast)  improve  search  rates 
among  non-targets  (shallower  slope  of  search  time  versus  number  of  items)  even  when  the 
operator  is  searching  for  more  than  a  single  attribute  (Treisman,  1982;  Treisman  &  Paterson, 
1984;  Treisman  &  Sato,  1990).  Even  more  interesting  is  the  finding  that  pre-attentive  search 
processes  can  respond  to  cognitive  as  well  as  perceptual  information.  For  example,  the  location 
of  a  novel  word  is  processed  more  rapidly  when  it  is  embedded  in  a  set  of  familiar  words  and 
digits  seem  to  “pop  out”  of  displays  that  contain  mostly  letters  (Johnston,  Hawley,  &  Famham, 
1993;  Teichner  &  Krebs,  1974).  A  related  phenomenon  is  the  “automatic”  processing  capability 
that  allows  humans  to  search  for  more  than  a  single  target  type  without  sacrificing  performance 
efficiency.  In  general,  each  additional  symbol  type  that  the  operator  searches  for  puts  additional 
demands  on  short-term  memory  (e.g.,  simultaneously  scanning  for  Taliban  strongholds  and 
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Northern  Alliance  defense  structures).  However,  if  the  relationship  between  potential  targets  and 
responses  is  extremely  well  learned,  then  searching  for  multiple  targets  simultaneously  is  not 
affected  by  the  additional  memory  requirement  (Schneider  &  Shiffrin,  1977).  Both  of  these 
phenomena  have  important  visualization  implications.  The  most  important  symbols  should  be 
chosen  carefully  to  be  easily  discriminable  from  other  display  symbols  (and  their  background), 
and  the  relationship  between  the  symbol  and  the  semantic  response  needs  to  be  consistent  and 
well  practiced.  A  good  situation  display  is  one  in  which  the  commander  can  notice  change 
instantly  and  be  able  to  keep  track  of  multiple  situations  concurrently  with  minimal  cognitive 
loading.  Other  cueing  techniques  such  as  blinking  and  sound  augmentation  can  be  used  to 
emphasize  important  changes,  but  in  general,  these  techniques  can  be  annoying  and  are  apt  to  be 
ignored  if  they  are  used. 

Of  all  the  codes  used  for  modem  symbology,  color  codes  seem  to  have  the  most  attractive 
qualities.  Color  coding  and  color  backgrounds  are  a  favorite  visualization  technique  to  segregate 
boundaries  and  to  code  various  important  tactical  situations.  However,  choosing  the  optimal 
color-coding  schema  is  not  trivial.  In  the  1930s,  the  International  Commission  on  Illumination 
(CIE)  developed  a  color  discrimination  schema  for  reflective  light.  More  recent  standards  have 
been  proposed  for  emissive  light  from  cathode  ray  tube  (CRT)  displays  (Carter,  1982).  The 
standards  have  been  modified  and  validated  with  human  performance  data,  which  indicate  that 
the  formula  derived  from  the  standards  (CIE  L*u*v)  is  a  good  predictor  of  human  discrimination 
performance  (Carter).  Algorithms  based  on  variations  of  the  formula  have  improved  the  engi¬ 
neer’s  ability  to  pick  color  palettes  that  are  maximally  discriminable  for  CRT  uses  (De  Corte, 
1986).  The  problem  is  that  it  is  difficult  to  use  a  formula  to  create  optimal  color  schemes  because 
many  military  symbols  have  been  assigned  colors  based  on  traditional  conventions  (red:  enemy 
or  danger,  blue:  friendly,  etc.),  and  map  backgrounds  are  so  variable.  A  related  problem  is  that 
the  number  of  color  codes  that  can  be  assigned  to  important  categories  is  limited  to  five  to  nine. 
Scenes  can  be  portrayed  in  a  nearly  infinite  array  of  color  gradations,  but  the  use  of  color  codes 
for  symbols  is  limited  by  human  working  memory  limitations;  too  many  codes  degrade  search 
performance  (Wickens  &  Hollands,  2000).  In  general,  color  coding  does  not  automatically 
increase  search  or  memory  performance,  especially  when  compared  to  well-learned  non-color 
codes.  In  fact,  there  are  important  instances  when  color  coding  leads  to  degraded  performance 
(Christ,  1975;  Christ  &  Corso,  1983;  Van  Orden,  Osga,  &  Luaben,  1991).  Usually,  problems 
were  found  when  the  test  subjects  had  to  ignore  color  codes  to  search  across  another  coding 
dimension.  Color  attracts  the  user’s  attention  and  it  interferes  with  his  or  her  ability  to  ignore 
color  codes  that  are  not  useful  for  a  particular  task.  Another  problem  is  that  a  portion  of  the  male 
population  has  problems  with  color  discriminations  and  to  make  matters  worse,  color  discrimi¬ 
nation  degrades  with  age.  However,  even  when  all  the  limitations  and  drawbacks  of  color  are 
considered,  most  researchers  agree  that  color  codes  enhance  situational  awareness  (Nugent, 
Keating,  &  Campbell,  1995;  Van  Orden,  Divita,  &  Shim,  1993). 
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Symbology  sets  are  used  successfully  in  many  civilian  applications  and  the  icons  that  represent 
the  various  applications  are  chosen  for  their  high  associative  value  with  the  represented  object 
(Collins,  1982).  It  has  even  been  suggested  that  pictograms  could  be  used  to  represent  a  formal 
universal  language  (Koler,  1969).  However,  symbology  has  been  most  successful  when  the 
applications  are  limited  and  the  symbol  set  is  extremely  intuitive.  Because  of  the  complexity  of 
military  applications,  militaiy  symbols  have  naturally  inflated  to  subsume  hundreds  of  different 
applications,  some  of  which  are  intuitive  and  many  of  which  simply  have  to  be  learned.  For 
example,  the  original  Navy  Tactical  Data  System  (NTDS)  symbols  were  designed  to  be  learned 
and  discriminated  easily.  However,  NTDS  became  unwieldy  as  more  applications  such  as  the 
Airborne  Early  Waming/Ground  Environment  Integration  Segment  (AEGIS)  class  cruisers  (and 
the  concomitant  symbols)  were  added.  Although  military  symbols  are  updated  periodically,  the 
basic  Army  symbols  have  remained  fairly  constant  (cf,  Department  of  the  Army,  1985).  The 
symbols  are  not  ideal  but  they  have  been  used  for  generations  and  would  require  extensive 
retraining  if  they  were  changed  substantially  (Weidenbacher  &  Barnes,  1997).  The  original 
symbol  sets  were  overlaid  on  military  maps  and  later  innovations  allowed  frequent  symbol 
changes  with  acetate  overlays  and  grease  pencils.  As  an  unexpected  bonus,  the  overlays  allowed 
the  commander  and  staff  to  review  the  trends  of  the  military  situation  by  flipping  through  the  old 
overlays.  New  North  Atlantic  Treaty  Organization  (NATO)  standards  (NATO,  1990)  have 
resulted  in  the  addition  of  more  human  friendly  symbology  and  the  adoption  of  color  standards. 
This  in  turn  has  influenced  the  United  States  to  coordinate  its  symbol  standards  across  services 
and  to  reformat  its  symbols  to  be  in  accord  with  human  engineering  standards  and  the  NATO 
standards  (Department  of  Defense,  1999).  Still,  the  symbol  sets  remain  fairly  static  for  the 
reasons  discussed  before.  The  introduction  of  digitized  displays  on  television  monitors  has 
precipitated  its  own  problems.  Many  of  the  display  surfaces  are  small,  causing  the  symbols  to 
cover  too  much  surface  area;  this  occludes  neighboring  symbols  and  gives  the  commander  only 
an  approximate  location  of  a  particular  unit.  Also,  the  color  contrast  between  a  symbol  and  its 
map  background  is  often  poor  because  the  color  contrast  among  emissive  displays  is  variable  and 
difficult  to  predict  for  all  the  possible  map  backgrounds  that  are  used.  In  general,  the  traditional 
military  symbology  conventions  are  less  than  ideal  for  modem  display  technology.  Data  suggest 
that  more  compact  symbols  with  greater  contrast  would  lead  to  improved  observer  performance 
(Weidenbacher  &  Barnes,  1997).  However,  traditional  symbol  conventions  are  ubiquitous  in 
military  systems.  The  software  replacement  and  training  costs  of  introducing  radically  new 
symbol  sets  would  most  likely  preclude  the  adoption  of  more  effective  concepts  in  the  near 
future. 

Future  situation  displays  will  be  multi-media  based  and  will  combine  knowledge  sources  from 
varied  commercial  and  military  sources.  The  U.S.  Navy  Space  and  Naval  Warfare  Systems 
Command  (SPAWAR)  system  center  is  in  the  process  of  developing  knowledge  wall  concepts  to 
replace  the  traditional  situation  maps  that  are  ubiquitous  in  modem  operations  centers.  The 
purpose  of  the  knowledge  wall  is  to  foster  shared  situational  awareness,  permit  continuous 
revision  of  the  military  situation,  and  enhance  the  senior  staffs  ability  to  interact  with  supporting 
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information  systems.  These  concepts  are  still  being  developed.  Figure  3  is  only  a  notional 
prototype  of  an  early  design  but  it  at  least  captures  the  variegated  information  sources  and  the 
live  multi-media  possibilities  of  the  actual  implementation.  Imagery  intelligence,  smart 
knowledge  portals,  media  from  anywhere  in  the  world,  video  of  ongoing  conflicts,  and 
conferencing  with  world  leaders  and  experts  are  some  of  the  obvious  possibilities  for  future 
knowledge  systems.  Based  on  extensive  interviews  with  30  senior  Naval  staff  officers,  the 
following  crucial  issues  (directly  quoted)  were  identified  as  important  characteristics  for  the 
knowledge  wall  (Oonk,  Smallman,  &  Moore,  2001): 

•  Shared  situation  awareness  among  its  users 

•  The  integration  of  relevant  mission  status  information 

•  An  intuitive  graphical  interface 

•  Consistently  formatted  information 

•  A  tactical  focus  for  the  displayed  information 

•  The  display  of  information  to  supplement  tactical  data 

•  The  display  of  mission  goals  and  Commander’s  Critical  Information  Requirements  (CCIR) 

•  The  display  of  summary  information  provided  by  “anchor  desk”  or  support  staff 

•  The  ability  to  connect  and  coordinate  or  collaborate  with  others  at  diverse  locations 

•  A  flexible  configuration  that  can  easily  be  changed  by  users 

•  The  ability  to  drill  down  through  displayed  information  for  more  detail 

•  Display  of  information  age  and  reliability 

•  Tactical  overlays  to  highlight  different  types  of  information 


Figure  3.  SPAWAR  knowledge  wall  concepts  (Oonk  et  al.,  2001). 


3.3  Abstract  Processes 

Ferren  (1999),  a  design  consultant  for  Walt  Disney’s  Imagineering  Division,  attacked  the  notion 
that  information  overload  was  inevitable  for  battlespace  visualization  displays  if  the  battle  pro¬ 
cess  were  represented  in  detail.  He  argued  that  the  problems  were  caused  by  poor  design  rather 
than  human  limitations.  His  thesis  is  that  symbols  and  icons  are  not  the  natural  manner  of  human 
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processing;  rather,  he  posits  that  if  information  could  be  presented  in  a  naturalistic  narrative 
format,  most  of  the  problems  associated  with  cluttered  situation  displays  could  be  circumvented. 
Many  of  the  design  principles  he  discussed  are  based  on  research  which  indicates  that  humans 
process  information  with  holistic  strategies  rather  than  feature  extraction  processes.  For  example, 
Pomerantz,  Sager,  and  Stoever  (1977)  found  what  they  referred  to  as  a  “configural  superiority 
effect”.  Their  subject’s  task  was  to  find  simple  physical  features  that  were  part  of  a  larger 
configuration  instead  of  trying  to  identify  the  same  features  in  isolation.  Counter-intuitively, 
adding  information  (as  part  of  a  configuration)  decreased  rather  than  increased  processing  time. 
Apparently,  individual  features  that  were  part  of  a  configuration  were  perceived  as  patterns  that 
formed  emergent  cues,  thus  enhancing  discrimination  performance  in  contrast  to  the  same  cues 
in  isolation.  A  similar  linguistic  phenomenon  was  discovered  which  showed  improved  recog¬ 
nition  of  a  letter  embedded  in  a  word  contrasted  with  the  same  letter  presented  alone  (Wheeler, 
1970).  The  design  implication  is  that  humans  are  particularly  efficient  at  recognizing  visual 
patterns  compared  to  processing  individual  objects.  This  design  principle  holds  for  strategically 
grouped  objects  as  well  as  a  single  configural  object.  For  example,  the  changing  visual  relation¬ 
ships  among  bar  graphs  can  be  used  as  the  emergent  cue  that  allows  humans  to  quickly  under¬ 
stand  higher  order  interactions  among  the  individual  indices  (Sanderson,  Flach,  Buttigieg,  & 
Casey,  1989). 

However,  an  unresolved  problem  is  that  humans  often  require  detailed  symbolic  information 
(unit  type,  readiness,  location,  mobility,  etc.),  and  understanding  patterns  by  themselves  is  not 
sufficient.  There  is  evidence  that  when  humans  attend  to  global  patterns,  they  miss  changes  in 
the  individual  components.  Bennett  and  his  colleagues  (Bennett  &  Flach,  1992;  Bennett,  Toms, 

&  Woods,  1993)  attempted  to  solve  this  problem  in  the  nuclear  power  plant  domain  by 
combining  principles  for  emergent  higher  order  features  with  basic  human  factors  design 
principles  to  highlight  individual  components.  The  configural  display  they  designed  depicted 
higher  order  process  features  related  to  overload  conditions  with  specific  details  of  the  individual 
components  (steam  and  water  flow  information)  annotated  on  the  same  display.  The  interactions 
among  the  components  were  displayed  in  terms  of  a  dynamically  changing  rectangle  whereas 
color-coded  sides  portrayed  measurement  data  for  individual  components.  Specifically,  changes 
in  area,  shape,  and  rate  of  change  of  the  rectangle  were  the  emergent  cues  that  signaled  important 
state  changes  in  the  power  plant  process.  After  practice,  the  operators  were  able  to  respond  to  the 
interaction  (emergent)  cues  without  losing  information  concerning  the  specific  component  cues. 
This  indicates  that  global  and  local  processing  cues  can,  at  least  in  certain  circumstances,  be 
attended  to  concurrently.  Barnes  and  Suantak  (1996)  used  the  same  configural  concept  to  design 
a  command  and  control  display  that  showed  the  relationship  between  combat  readiness  and 
mobility  information  of  maneuver  battalions  with  an  integrated  configural  representation. 
Empirical  results  (with  16  officers  at  Fort  Huachuca,  Arizona)  indicated  that  configural 
representations  resulted  in  more  effective  situational  awareness  and  simple  tactical  decisions 
than  current  Army  display  symbology. 


11 


Possibly,  the  most  widely  used  abstract  representation  is  the  spatial  metaphor.  The  idea  is  simple: 
display  entities  are  arranged  in  space  to  show  their  relationships  in  the  problem-solving  domain 
of  interest.  For  example,  a  spreadsheet  is  arranged  in  some  temporal  and  categorical  order  to 
represent  financial  trends  and  relationships.  For  complex  databases,  the  problem  is  to  represent 
the  data  in  such  a  manner  as  to  capture  the  local  relations  without  losing  the  big  picture.  A 
number  of  techniques  have  been  suggested  to  visualize  the  abstract  relationships  among  data 
clusters.  Two-dimensional  approaches  use  “fish  eye”  techniques  that  emphasize  the  most 
important  current  data  clusters  by  displaying  them  in  central  locations,  whereas  other  data 
relations  cover  smaller  areas  on  the  periphery.  There  are  also  a  number  of  techniques  that  use 
cone  representations,  mathematical  proximity  functions,  and  various  navigational  and  browsing 
utilities  for  understanding  3-D  databases  (Wickens  &  Hollands,  2000;  Fairchild,  Poltrock,  & 
Furnas,  1988).  An  example  of  the  mathematical  techniques  is  “Pathfinder,”  which  generates  a 
network  representation  based  on  cognitive  distances  derived  from  expert  judgments  that  use 
psychometric  scaling  techniques  (Minkowski  r-metric).  Pathfinder  and  similar  proximity 
functions  can  be  used  to  display  various  cognitive  relationships  among  data  sources  via  map 
metaphors  (McDonald  &  Schvanveldt,  1988).  Such  techniques  are  not  without  their  drawbacks; 
there  are  no  unique  cognitive  distance  functions1,  and  these  functions  may  vary  a  great  deal 
among  experts.  However,  the  military  is  beginning  to  use  proximity  functions  to  organize  large 
intelligence  databases  to  allow  the  analyst  to  “see”  the  big  picture  and  to  focus  on  similarity 
clusters  that  are  pertinent  to  the  current  intelligence  requirements. 

On  a  more  abstract  level,  Healy,  Booth,  and  Enns  (1998)  demonstrated  the  use  of  pre-attentive 
cues  (curvature  and  color)  for  efficient  visual  parsing  of  multivariate  data  sets  (cf,  Treisman, 
1982).  More  recently,  Healy  and  Enns  (1999)  combined  textural,  height,  and  color  cues  to  design 
complex  oceanographic  displays  that  show  shifting  patterns  of  plankton  density,  current  strength, 
and  sea  surface  temperature.  In  particular,  they  were  interested  in  the  combination  of  color  and 
textural  cues  as  independent  dimensions  to  code  various  oceanographic  phenomena  such  as 
typhoon  development.  They  found  that  a  search  for  color-coded  information  was  unaffected  by  a 
background  of  textural  codes  (density,  size,  height,  regularity,  etc.)  but  that  there  was  a  cost  for 
searching  for  textural  coded  targets  in  a  multi-hued  display.  Most  of  the  search  times  were 
extremely  rapid,  which  suggested  again  that  color  and  textural  codes  could  be  processed  pre- 
attentively.  Perhaps  the  more  interesting  question  of  whether  higher  order  emergent  features 
could  be  abstracted  from  their  displays  was  not  addressed.  For  example,  it  would  have  been 
fascinating  to  know  whether  visualization  patterns  representing  precipitation,  wind,  and  ocean 
current  cues  and  their  interactions  could  be  used  to  predict  incipient  weather  changes.  As  Ferren 
(1999)  noted,  the  human  visualization  system  is  able  to  extract  complex  cues  to  understand 
important  trends  in  our  often  volatile  environment.  Precisely  how  our  perceptual  system 


’For  example,  during  the  cold  war,  the  United  States  and  the  Soviet  Union  were  perceived  as  being  close  in  terms  of 
military  power  but  distant  in  terms  of  political  philosophy. 
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accomplishes  this  feat  is  still  mostly  a  matter  of  empirical  investigation  rather  than  theoretical 
understanding. 


4.  Prediction 


4.1  Visualization  and  Decision  Making  During  Uncertainty 

Because  many  military  situations  are  inherently  uncertain,  displaying  risk  and  associated 
concepts  such  as  uncertainty  and  utility  are  important  issues  for  FCS.  There  is  a  tradition  of 
psychological  research  related  to  risky  decisions  that  is  still  lively  and  controversial  (cf, 
Gigerenzer  &  Hoffrage,  1999;  Klein,  1999;  Lopes,  1986).  One  school  of  thought  emphasizes 
decision  biases  and  the  non-rational  aspects  of  human  decision  making.  Certainly,  the  type  and 
severity  of  the  cognitive  biases  reported  in  the  decision-making  literature  are  extensive  (Wickens 
&  Hollands,  2000).  For  example,  in  an  Army-sponsored  project,  researchers  found  that  nine 
types  of  cognitive  biases  had  important  impacts  on  the  way  intelligence  analysis  is  performed 
(Thompson,  Hopf-Weichel,  &  Geiselman,  1983).  Much  of  the  research  suggests  that  humans  do 
not  compute  probabilities  among  an  exhaustive  set  of  options;  instead,  they  visualize  concrete 
examples  and  use  frequency  judgments  to  assign  uncertainty  values.  Thus,  many  of  the  decision 
biases  are  a  result  of  the  heuristic  processes  that  humans  use  to  visualize  the  problem  space 
(Johnson-Laird,  Legrenzi,  Girotto,  Legrenzi,  &  Cavemi,  1999).  The  U.S.  Army  Research 
Laboratory  (ARL),  working  with  the  University  of  Illinois,  investigated  the  effects  of  these 
biases  and  possible  visualization  techniques  to  mitigate  them  in  a  series  of  missile  defense 
simulations.  In  the  initial  experiment.  Air  Force  officers  acted  as  operators  defending  against  a 
national  missile  attack.  Figure  4  shows  the  visualizations  for  the  simulation.  The  left-hand  screen 
showed  the  trace  history  of  the  incoming  missiles  and  the  inventory  of  ground-based  interceptors 
(GPIs).  The  right-hand  top  display  portrayed  the  status  of  the  incoming  missiles  and  their 
probable  targets.  The  bottom  right-hand  display  indicated  the  GPI  allocation  scheme  for  each 
incoming  missile.  Periodically,  the  computer  showed  the  uncertainties  associated  with  successful 
defense  (or  possible  loss)  of  the  defended  cities,  based  on  the  current  number  of  incoming 
missiles  and  remaining  inventory  of  interceptor  missiles. 

The  first  experiment,  which  used  16  trained  national  missile  defense  (NMD)  potential  operators 
(the  system  is  still  being  developed),  investigated  different  formats  for  displaying  uncertainty. 
The  NMD  approach  at  the  time  of  the  experiment  was  to  display  probability  of  mission  success 
(overall  and  for  each  defended  target),  which  proved  to  be  abstract  and  somewhat  confusing 
because  of  the  high  degree  of  certitude  required  in  such  cases.  The  researchers  displayed  the 
same  information  in  terms  of  expected  frequency  of  enemy  “leakers”  (the  expected  rate  of  enemy 
missiles  not  intercepted),  arguing  that  the  expected  frequency  format  was  more  concrete  and 
more  easily  visualized  than  abstract  probabilistic  representations  (cf,  Gigerenzer  &  Hoffrage, 
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1999).  The  operators’  overall  situational  awareness  did  improve  with  the  concrete  “leaker” 
format  in  terms  of  superior  memory  and  search  performance.  However,  the  operators’  decisions 
whether  to  add  more  reserve  missiles  to  the  current  mission  (thus  leaving  defended  cities 
vulnerable  to  an  ensuing  attack)  were  not  affected  by  the  uncertainty  format,  which  indicated  that 
no  bias  was  introduced  by  the  “leaker”  metric.  There  was  an  interaction  between  current  risk  and 
the  probability  of  an  ensuing  attack.  Basically,  operators  were  responsive  to  the  possibility  of  an 
ensuing  attack  but  they  were  much  more  concerned  about  the  known  risks  involved  in  the  current 
mission.  The  results  of  this  experiment  suggest  that  a  concrete  representation  of  uncertainty  in 
terms  of  expected  frequencies  of  “leaker”  missiles  is  a  promising  format  for  aiding  operators  in 
visualizing  uncertainty,  but  more  research  needs  to  be  done  to  generalize  beyond  the  specific 
task  used  for  this  experiment.  What  is  particularly  interesting  is  the  fact  that  the  visualization 
improvement  was  not  the  result  of  pictorial  or  graphical  variables  but  involved  a  simple  change 
in  numerical  format  that  allowed  the  operator  to  visualize  risk  in  terms  of  concrete  examples 
rather  than  abstract  concepts.  There  is  also  a  disturbing  suggestion  that  operators  may  focus  too 
heavily  on  the  immediate  fight  to  the  detriment  of  planning  for  future  engagements,  which 
suggests  that  visualization  factors  related  to  future  shortages  of  interceptor  missiles  need  to  be 
investigated. 


Figure  4.  Risk  management  visualizations  for  missile  defense  simulations  (Bames,  Wickens,  &  Smith, 
2000). 


Another  series  of  studies  investigated  biases  related  to  how  military  personnel  visualize 
sequences  of  hits  and  misses  of  interceptor  missiles  against  incoming  missiles  (Smith  & 
Wickens,  1999).  In  general,  humans  tend  to  visualize  independent  stochastic  processes  using 
easily  imaged  heuristics  such  as  things  getting  better  (or  worse)  or  they  use  anchoring  stratagems 
to  perceive  trends  that  do  not  exist.  For  example,  test  subjects  tend  to  see  trends  in  data  that  are, 
in  fact,  stochastically  independent  even  when  they  are  informed  of  the  mathematical  relation- 
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ships  (Wickens  &  Hollands,  2000).  In  two  separate  experiments,  Patriot  missile  operators 
perceived  nonexistent  trends,  anchoring  their  decisions  to  the  first  or  last  piece  of  information 
they  received,  which  indicated  individual  differences  as  well  as  effects  caused  by  slight  changes 
in  experimental  conditions  (Adelman  &  Bresnick,  1992;  Adelman,  Bresnick,  Black,  Freeman,  & 
Sak,  1996).  In  a  second  NMD  experiment,  16  Reserve  Officer  Training  Corps  students  and  4 
active  duty  officers  stationed  at  the  University  of  Illinois  were  used  to  investigate  trend  effects 
related  to  observing  the  sequence  of  hits  or  misses  of  interceptors  against  the  incoming  enemy 
missiles.  The  trend  data  were  investigated  in  the  simulation  game  mentioned;  the  participants 
were  informed  of  the  stochastically  independent  properties  of  hits  and  misses  in  the  simulation. 
As  in  the  Patriot  case,  the  trend  effects  were  complex.  In  particular,  the  perception  of  things 
getting  worse  caused  the  operators  to  over-react  and  take  more  missiles  out  of  reserve  than  they 
would  for  an  equivalent  case  when  the  hits  and  misses  seem  to  be  of  a  more  random  nature 
(Barnes,  Wickens,  &  Smith,  2000).  In  the  real  world,  trends  are  often  of  the  type  where  events 
start  to  improve  or  deteriorate.  The  problem  posed  here  is  how  to  analyze  and  then  present 
information  to  the  operator  so  that  the  difference  between  random  fluctuations  and  actual  battle 
trends  can  be  visualized.  It  is  worth  noting  that  the  problems  investigated  for  the  NMD  will 
surface  for  many  of  the  new  automated  systems  being  developed  for  military  intelligence,  Patriot 
and  other  missile  defense  systems,  and  sensor-to-shooter  field  artillery  systems  (Thompson  et  al., 
1983).  As  evolving  systems  become  more  sophisticated,  the  display  of  states  of  uncertainty  and 
the  concomitant  cognitive  biases  will  require  innovative  cognitive  engineering  solutions. 
Visualization  techniques  and  more  effective  ways  of  portraying  uncertainty  are  at  least  part  of 
the  solution. 

4.2  Naturalistic  Decision  Making  and  Mental  Simulation 

Whereas  the  previous  reported  research  focused  on  the  negative  aspects  of  human  decision 
making  and  their  heuristic  processes,  many  researchers  have  pointed  out  the  efficiencies  of  the 
same  processes.  The  laboratory  research  has  been  criticized  as  being  unrealistic  and  unrelated  to 
tasks  that  humans,  especially  experts,  usually  perform.  In  particular,  a  number  of  researchers 
have  questioned  the  generality  and  the  construct  validity  of  research  that  was  not  conducted  in 
more  naturalistic  situations  (Klein,  1999;  Lopes,  1986).  Much  of  the  early  research  about 
decision  making  assumed  a  normative  process  underlying  human  decision  making.  Humans 
deviated  from  this  process  but  they  could  be  trained  to  use  prescriptive  techniques  that  would 
enhance  their  decision-making  skills.  This  approach  influenced  U.S.  Army  doctrine,  causing  the 
Command  and  General  Staff  College  at  Fort  Leavenworth,  Kansas,  to  train  staff  officers  to 
evaluate  three  courses  of  action  (COAs)  and  enumerate  their  strengths  and  weaknesses  (and  the 
inherent  uncertainties  involved)  before  making  a  recommendation  to  the  commander.  This 
paradigm  is  very  similar  to  the  traditional  structured  approach  based  on  the  tenets  of  decision 
analytical  theory  being  taught  at  most  university  business  schools.  The  theoretical  implication  is 
that  the  successful  decision  maker  needs  to  know  what  options  are  available,  the  uncertainties 
and  utilities  involved,  and  the  general  structure  of  the  decision  space  (usually  some  sort  of  a 
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decision  tree)  before  he  or  she  chooses  an  optimal  solution.  The  principal  weakness  of  this 
paradigm  is  that  it  has  little  to  do  with  how  experts  actually  make  decisions.  For  example,  Klein, 
in  a  now  classic  study,  found  that  expert  firemen  did  not  generate  options  but  simply  “knew”  the 
correct  thing  to  do.  Klein  found  the  same  basic  process  for  experienced  business  executives  and 
military  officers.  Experts,  especially  in  time-constrained  situations,  focus  on  situational  under¬ 
standing  and  then  make  decisions  rapidly,  usually  without  considering  options.  Klein  refers  to 
this  process  as  recognition-primed  decision  making  (RPD).  Experts  “see”  familiar  patterns  and 
visualize  solutions  using  external  cues  that  trigger  memories  of  successful  actions  for  similar 
events.  The  evolutionary  advantage  is  obvious;  primitive  man  had  scant  opportunities  to 
construct  decision  trees.  Klein  does  not  assume  that  all  decision  making  is  this  rapid;  in  more 
ambiguous  situations,  the  expert  may  visualize  various  scenarios  and  CO  As,  thus  creating  a 
mental  simulation  of  the  possibilities  and  their  consequences  before  making  a  decision. 

The  key  to  military  decision  making  is  not  a  normative  process  but  the  warfighter’s  ability  to 
visualize  a  solution  that  is  highly  dependent  on  past  experience  and  situational  understanding. 
Kobus,  Proctor,  Bank,  and  Holste  (2000)  investigated  52  marines  making  COA  decisions  in  a 
dynamically  changing  military  situation  during  both  high  and  low  uncertainty  conditions.  The 
most  important  differences  were  between  experienced  and  inexperienced  marines.  The  results 
indicated  that  experienced  marines  took  longer  to  make  their  situational  assessment  but  made 
COA  decisions  more  accurately  and  rapidly  than  inexperienced  marines.  The  experienced 
marines  were  also  less  affected  by  the  degrees  of  uncertainty  concerning  the  validity  of  the 
intelligence  information  they  received.  The  advantage  of  experience  seem  to  be  that  it  allowed 
the  officers  to  visualize  an  effective  COA  rapidly  once  enough  knowledge  was  digested  for  them 
to  understand  the  military  situation.  Also,  other  research  confirms  that  introducing  uncertainty 
has  a  significantly  more  deleterious  effect  on  inexperienced  military  decision  makers  than  on 
their  more  experienced  peers  (St.  John,  Callan,  Proctor,  &  Holste,  2000). 

In  a  different  domain,  researchers  found  that  the  ability  of  an  expert  to  adapt  to  new  problem  sets 
depends  on  a  rich  interconnectedness  among  the  expert’s  knowledge  structures,  allowing 
generalization  to  the  problem  space.  Experts  are  not  necessarily  more  logical  than  the  novice  but 
have  more  stored  patterns  (and  their  stored  relationships  to  other  patterns)  to  compare  to  the  new 
situation.  For  example,  researchers  found  important  differences  in  problem-solving  abilities 
between  experts  and  less  experienced  Unix2  programmers.  Experts  were  more  adept  at 
combining  Unix  commands  to  solve  novel  programming  problems  than  were  programmers  with 
less  than  2  years’  experience.  Experts  were  able  to  visualize  multi-step  solutions,  whereas  the 
relative  novices  were  still  at  the  individual  software  command  level  (Mannes  &  Doane,  1991; 
Doane,  Alderton,  Sohn,  &  Pelligrino,  1996;  Doane,  Sohn,  McNamara,  &  Adams,  2000). 

Holy  oak  and  Spellman  (1993)  point  out  that  solving  new  problems  requires  more  than  RPD 
processes.  Humans  must  not  only  retrieve  past  memories  but  must  also  mix  and  match  them  to 
the  current  problem  until  a  new  solution  emerges.  The  thought  processes  themselves  tend  to  be 

2Unix®  is  a  registered  trademark  of  The  Open  Group. 
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“imagery”  based  and  to  depend  on  the  investigation  of  more  likely  solutions  rather  than  on  an 
assessment  of  all  possible  solutions  (Johnson-Laird  et  al.,  1999).  The  research  results  suggest 
that  training  via  visualization  techniques  and  role  playing  in  various  military  vignettes  may  aid  in 
the  development  of  the  “expertise”  to  adapt  to  new  environments.  Another  product  of  naturalistic 
approaches  involves  a  better  understanding  of  how  humans  tend  to  use  visualizations  in  real- 
world  situations.  Interviews  with  experts  suggest  that  humans  visualize  in  terms  of  analogies  to 
past  experiences,  making  metaphors  and  narratives  effective  forms  of  exposition.  The  implica¬ 
tion  is  that  the  knowledge  presentation  should  tell  a  story  because  a  narrative  presentation 
involves  observers  in  a  mental  dialogue  with  the  presentation  material,  thus  allowing  them  to  fill 
the  blanks  (Gershon  &  Page,  2001).  Commanders  do  not  need  to  be  overwhelmed  with  details 
but  need  to  be  presented  only  with  that  part  of  the  story  that  directly  impacts  their  decisions 
(Ferren,  1999;  Klein,  1999).  This  is  an  important  contrast  to  the  traditional  wisdom  of  how  to 
present  information.  Humans  were  modeled  as  passive  information  receivers  who  at  best  could 
decode  all  the  information  on  the  display  (cf,  information  theory  in  Wickens  &  Hollands,  2000). 
In  the  naturalistic  view,  humans  are  active  creators  of  information,  based  on  their  past  experience 
and  their  ability  to  understand  the  current  situation  using  analogical  reasoning  and  metaphorical 
examples.  As  intuitively  appealing  as  the  naturalistic  view  is,  there  are  obvious  drawbacks.  It  is 
difficult  to  scientifically  define  “storytelling”  formats  and  to  show  that  narration  improves 
situational  awareness  or  decision  making  more  than  well-designed  information  displays  (or  more 
precisely,  during  what  conditions  it  does  or  does  not  do  so).  The  research  implications  are  that 
we  need  to  collect  data  and  evaluate  concepts  in  realistic  military  environments.  However,  this  is 
a  difficult  and  not  easily  accomplished  task.  Too  often,  data  collected  in  realistic  exercises  are 
not  useful  because  of  the  complete  lack  of  controls.  A  more  measured  approach  would  be  to 
combine  the  strengths  of  controlled  experiments  with  the  realism  of  field  work  via  a  hybrid 
research  program  (Barnes  &  Beevis,  in  press).  The  same  is  true  in  investigating  human  biases  as 
opposed  to  expert  strengths.  It  is  a  matter  of  a  glass  half  full  or  half  empty;  both  phenomena 
exist,  and  any  realistic  research  effort  must  investigate  the  warfighter’s  limitations  as  well  as 
strengths.  However,  we  do  need  to  focus  our  efforts  more  on  using  realistic  simulation  and 
experimental  paradigms  that  capture  the  important  issues  of  specific  military  domains  if  we  are 
going  to  understand  these  phenomena  in  the  correct  military  context.  Eventually,  the  resulting 
concepts  must  be  validated  dming  actual  military  exercises. 

4.3  Visualization  Issues  for  Supervisory  Control  of  Automated  Battlefields 

An  important  question  in  future  combat  environments  is  how  much  trust  the  commander  will 
have  in  automated  systems.  Commanders  will  never  relinquish  their  decision  prerogatives  even 
though  processing  and  execution  will  become  increasingly  automated.  The  commander  must  be 
able  to  understand  and  visualize  the  consequences  of  each  of  these  systems  as  they  execute  on 
the  battlefield  in  order  to  achieve  his  or  her  intended  end  states.  Trust  does  not  mean  that  the 
commander  will  assume  that  systems  will  operate  perfectly.  For  the  foreseeable  future,  an 
operator  will  be  in  the  loop  at  a  supervisory  level.  Rather,  appropriate  trust  is  an  ability  to 
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visualize  how  automated  systems  will  operate  in  a  specific  military  environment  and  when  to 
intervene  with  redundant  systems.  At  the  operator  level,  appropriate  trust  in  automated  solutions 
will  become  an  increasingly  important  component  of  executing  the  commander’s  intent.  Early 
research  is  pessimistic,  indicating  many  cases  of  misuse  (over-reliance  on  automated  systems) 
and  disuse  (over-reliance  on  manual  solutions)  (Parasuraman  &  Riley,  1997).  The  results  can  be 
disastrous:  airplane  crashes  when  pilots  relied  too  much  on  automated  landing  systems,  train 
crews  ignoring  the  speed  regulators  leading  to  rail  accidents,  or  the  infamous  K-007  Korean 
aircraft  incident  when  the  crew  ignored  evidence  of  route  deviation  once  the  automatic  pilot  was 
set  (miss-set).  The  problem  is  fuzzy  because  there  seem  to  be  biases  toward  relying  too  little  and 
too  much  on  automated  systems.  Researchers  sponsored  by  ARL  at  Fort  Sill,  Oklahoma,  have 
done  a  number  of  studies  investigating  trust  issues  in  Army-related  tasks.  They  have  varied 
motivational,  social,  and  cognitive  factors  and  they  found  examples  of  misuse  and  disuse  of 
automated  systems  (Dzindolet,  Pierce,  Beck,  &  Dawe,  2002). 

In  a  series  of  experiments  using  a  simulated  automatic  target  recognition  (ATR)  device, 
Dzindolet  et  al.  found  that  the  test  subjects  were  more  willing  to  rely  on  their  own  estimates  even 
when  they  were  informed  that  the  ATR  solutions  were  superior  to  their  own,  based  on  the  last 
200  trials.  Subjects  were  also  more  willing  to  trust  the  same  estimates  if  they  were  told  that  they 
came  from  a  peer.  Thus,  subjects  in  this  series  of  experiments  showed  a  bias  against  automation; 
some  of  the  participants  even  suggested  that  they  had  a  moral  imperative  to  trust  their  own 
judgment  over  a  machine's  (Dzindolet  et  al.,  2002).  However,  another  study  using  a  similar 
paradigm  (Dzindolet,  Pierce,  Beck,  Dawe,  &  Anderson,  2001)  indicated  a  complete  shift  in 
strategy,  with  the  participants  more  prone  to  misuse  than  to  disuse.  They  tended  to  over-rely  on 
the  aid  when  the  aid  was  incorrect.  The  crucial  factor  was  the  order  of  the  decision  making.  In 
the  latter  experiment,  the  machine’s  solution  was  displayed  before  the  operator  made  a  decision, 
whereas  in  the  former  experiments,  the  operator’s  decision  preceded  the  machine’s.  A  compari¬ 
son  of  the  experiments  indicates  that  the  shift  from  a  manual  bias  (“I  make  the  decisions”)  to  an 
automation  bias  (“let  the  machine  do  it”)  is  based  on  type  of  feedback  and  time  stress  but  mainly 
by  whether  the  operator  had  already  made  a  decision  before  the  machine  solution  was  known. 
The  results  are  important  because  it  gave  insight  into  the  dynamics  of  automation  bias,  which  has 
some  of  the  characteristics  of  social  loafing  (if  the  machine  has  already  made  the  decision,  the 
operator’s  uncertainty  could  be  resolved  simply  by  agreeing).  However,  if  operators  have  already 
set  their  minds  by  the  time  they  see  the  automated  decision,  then  they  have  a  vested  interest  in 
following  their  own  dictates.  Again,  it  is  an  example  of  a  very  simple  manipulation,  apparently 
changing  the  operator’s  visualization  of  the  problem  space  from  one  where  the  operator  is  the 
controller  and  the  computer  is  the  assistant  (under-reliance)  to  one  where  the  computer  is 
authority  and  the  operator  the  backup  (over-reliance). 

Ensuing  experiments  indicated  that  the  precision  of  the  feedback  as  well  as  the  decision  order 
was  important  in  instilling  appropriate  trust  in  the  operator.  An  experimental  condition  in  which 
the  operator  was  given  feedback  about  each  trial  and  when  obvious  errors  were  not  displayed  to 
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the  operator  resulted  in  nearly  optimal  operator  automation  decisions.  In  this  condition,  the 
operator  was  able  to  visualize  a  rational  strategy  in  regard  to  the  machine  error  rate  and  was  able 
to  intervene  appropriately  (Pomranky,  Dzindolet,  &  Peterson,  2001).  One  issue  that  became 
obvious  in  this  series  of  experiments  and  the  NMD  experiments  is  the  importance  of  training  as 
well  as  improvements  in  visualization  technology.  The  Pomranky  et  al.  experiment  showed  that 
optimal  synergy  depended  on  the  operator  learning  the  characteristics  of  the  automated  device. 
The  displayed  information  is  only  meaningful  if  combined  with  training  with  feedback  that 
stresses  consequences  (“you  just  missed  an  obvious  target”).  Visualization  depends  on  memory 
of  past  incidences  combined  with  cues  relating  to  the  current  situation.  When  these  cues  are 
properly  understood,  the  operator  can  visualize  his  or  her  appropriate  role  within  the  automated 
system. 

Another  important  issue  related  to  supervisory  control  is  the  degree  of  automation  and  reliability 
problem  (Parasuraman,  Sheridan,  &  Wickens,  2000).  FCS  will  depend  on  unmanned  aerial  and 
ground  robotic  systems  with  varying  amounts  of  autonomy.  The  crucial  issue  for  these  systems  is 
how  much  autonomy  to  assign  to  the  unmanned  systems — a  question  that  depends  on  the  opera¬ 
tor’s  situational  awareness  as  well  as  the  reliability  and  degree  of  automation  of  the  unmanned 
systems.  For  example,  it  may  be  beneficial  from  a  systems  point  of  view  to  keep  operators 
actively  engaged  in  the  control  of  a  semi-autonomous  vehicle  because  it  forces  them  to  maintain 
good  situational  awareness  for  those  instances  when  the  operator  is  truly  needed.  Complacency 
effects  are  a  problem  even  when  the  automation  is  implemented  only  at  the  information  level. 
Horrey  and  Wickens  (2001)  found  benefits  and  costs  of  partially  automating  military  situation 
displays.  Overall,  automated  highlighting  and  the  presentation  of  diagnostic  information  con¬ 
cerning  enemy  movements  aided  military  decision  making;  however,  their  subjects  were  misled 
by  unreliable  automation,  causing  them  to  miss  important  cues  that  may  have  been  spotted  in 
manual  conditions.  Galster,  Bolia,  Roe,  and  Parasuraman  (2001)  found  the  same  trends  using  a 
military  aviation  display.  More  intriguing  is  the  “automation  paradox”  effect  found  in  a  number 
of  military  and  applied  experiments.  Researchers  found  that  the  more  reliable  the  automation,  the 
more  likely  the  operator  was  to  make  mistakes  when  the  automation  proved  unreliable  (Rovira, 
Zinni,  &  Parasuraman,  2002).  This  suggests  that  the  engineering  solution  of  designing  more 
reliable  systems  has  its  own  human  performance  costs.  Especially  for  more  reliable  aids,  humans 
can  become  too  complacent  and  fail  to  anticipate  automation  failures  (even  when  they  know  they 
can  occur)  (Rovira  &  Parasuraman,  2002). 

Trust  is  a  pervasive  combat  issue  that  affects  all  command  decisions.  In  the  future,  the  issues  will 
not  only  involve  subordinate  commanders  and  their  individual  characteristics  but  also  subordi¬ 
nate  intelligent  systems  and  their  characteristics.  The  most  important  research  issues  involve 
determining  the  cognitive  engineering  paradigms  that  best  allow  the  commander  to  understand 
the  operations  of  automated  systems  so  that  the  degree  of  autonomy  and  trust  for  systems  is  a 
natural  and  effective  part  of  the  overall  command  process.  Some  combination  of  training  and 
improved  visualization  concepts  is  required  so  that  the  commander  (and  his  or  her  operator 
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surrogates)  can  understand  and  predict  automated  performance.  The  research  about  the 
automation  paradox  and  complacency  also  implies  that  it  is  important  to  engage  the  operator  in 
the  decision  process  even  for  completely  automated  systems.  However,  we  need  to  know  more 
about  the  degree  of  autonomy,  complacency,  feedback  effects,  and  the  automation  paradox  in 
realistic  military  settings  before  we  can  design  effective  visualization  aids. 

4.4  Visualization  and  Prediction  in  Complex  Military  Environments 

Perhaps  the  most  influential  scientific  change  in  the  20th  century  is  a  shift  from  a  belief  in  a 
deterministic  Newtonian  world  to  a  scientific  Zeitgeist  that  accepts  the  notion  of  inherent  uncer¬ 
tainties.  More  recently,  even  the  notion  of  a  being  able  to  make  probabilistic  predictions  has  been 
challenged.  Chaos  theory,  for  example,  assumes  that  minor  events  can  radically  alter  the 
outcome  of  nonlinear  processes  in  a  world  with  many  such  processes.  Complexity  theory  is  a 
group  of  related  concepts  that  is  predicated  on  two  important  concepts:  information  and  emer¬ 
gence.  The  first  involves  the  overwhelming  number  of  possible  outcomes  in  any  large  set  of 
interacting  parts.  The  second  involves  the  result  of  the  possible  interactions  in  a  complex  system. 
For  example,  there  are  2"  possible  states  for  a  system  with  n  components.  For  any  one  of  the 
myriad  interactions,  it  is  possible  for  unpredictable  behaviors  to  “emerge”  that  can  have  minor  or 
major  impacts  on  the  system  as  a  whole  (Bar-Yam,  1997;  Waldrop,  1992).  Considering  the 
battlefield  as  a  complex  system,  complexity  theory  implies  that  the  dynamics  of  battlefield 
skirmishes  become  unpredictable  as  the  number  of  interacting  elements  increase.  This  is  true 
even  in  a  successful  battle.  For  these  situations,  the  commander  (through  training,  doctrine,  and 
tactical  redundancy)  has  accounted  for  unexpected  events,  and  the  resulting  divergence  from  the 
battle  plan  has  only  minor  impacts.  However,  as  the  inherent  uncertainty  of  the  situation 
increases,  “emergence”  of  unexpected  events  and  chaotic  behaviors  will  start  to  play  an 
increasingly  important  role  in  determining  the  outcome. 

Unfortunately,  future  military  missions  will  tend  to  be  the  type  for  which  doctrine  and  precon¬ 
ceived  notions  of  war  fighting  will  be  challenged  in  many  different  situations.  In  particular, 
opposition  forces  have  shown  the  ability  to  adapt  to  western  preconceptions  and  to  use  asym¬ 
metric  tactics  to  defeat  superior  force  and  firepower,  especially  when  political  and  psychological 
factors  are  being  exploited.  Complexity  and  the  effects  of  unforeseen  interactions  have  become 
topics  of  intense  debate  among  military  researchers.  However,  empirical  and  applied  research  in 
this  area  is  still  in  the  beginning  stages.  Currently,  ARL  is  investigating  new  prediction  and 
visualization  paradigms  to  enhance  the  commander’s  situation  understanding  in  these  environ¬ 
ments.  New  visualization  techniques  have  been  proposed  to  help  us  understand  the  historical 
trends,  political  changes,  ethnic  conceptions,  and  changing  perceptions  of  various  combatant  and 
non-combatant  groups  in  a  particular  area  of  concern  (Zacharias  &  Hudlicka,  2001).  One  area  of 
particular  concern  for  missions  in  unfamiliar  environments  is  the  use  of  information  aids  to  assist 
the  intelligence  analyst  in  interpreting  the  streams  of  data  that  can  overwhelm  military  capacity 
during  critical  situations.  University  of  Illinois  researchers  in  support  of  ARL  have  developed  a 
series  of  decision  support  systems  using  Bayesian  belief  nets  that  can  be  used  as  tools  for  an 
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individual  analyst  or  in  a  collaborative  environment  to  structure  and  categorize  incoming 
intelligence  reports  (Mengshoel  &  Wilkens,  2001;  Asaro,  Hayes,  &  Jones,  2001).  The  results  are 
promising  but  the  systems  are  still  very  much  laboratory  tools.  The  main  problem  is  that  the 
probability  structure  must  be  created  to  account  for  each  new  battle  situation.  The  advantage  of 
these  systems  is  that  they  handle  large  data  streams  and  show  inferential  networks  to  the  trained 
analyst,  which  will  allow  him  or  her  to  visualize  the  emerging  trends.  More  research  needs  to  be 
done  to  determine  how  robust  these  systems  are  when  the  military  situation  changes.  They  show 
promise  in  helping  to  ameliorate  an  important  consequence  of  complexity:  the  overwhelming 
amount  of  battlefield  information. 

Another  approach  to  predicting  outcomes  on  the  battlefield  is  the  use  of  genetic  algorithms  to 
“search”  dynamically  for  the  underlying  structure  by  rapidly  comparing  possible  solutions.  The 
solutions  are  generated  by  the  combination  of  important  characteristics  (“genes”)  to  examine 
hundreds  of  thousands  of  options  with  surprisingly  little  computer  overhead.  The  FOX3  genetic 
algorithm  (GA),  which  was  crafted  at  the  University  of  Illinois,  was  developed  to  generate  and 
evaluate  solutions  to  tactical  problems  in  conventional  warfare  environments.  The  algorithm, 
which  is  combined  with  a  display  that  portrays  the  results,  assisted  surrogate  commanders  from 
Fort  Leavenworth,  Kansas,  by  helping  them  visualize  twice  as  many  new  CO  As  as  they  could 
with  the  manual  planning  condition  (Hayes,  Fiebig-Brodie,  Winkler,  &  Schlabach,  2001).  Hillis 
and  Winkler  (2001)  have  taken  the  genetic  algorithm  concept  one  step  further  by  introducing  co¬ 
evolutionary  solutions  to  FOX-GA.  This  approach  allows  the  analyst  to  observe  multiple  itera¬ 
tions  of  the  genetic  solution  by  playing  the  red  and  blue  force  best  solutions  against  each  other. 
The  tactics  of  red  and  blue  co-evolve  as  the  best  solution  in  game  n-1  is  challenged  by  the  adver¬ 
sary  in  game  n  from  which  the  best  solution  then  determines  the  initial  point  of  game  n+1,  etc. 
The  advantage  is  that  solutions  emerge  that  would  go  undetected  in  a  simpler  one-way  game 
such  as  the  original  FOX-GA.  The  empirical  question  is  whether  the  added  complexity  aids  or 
hinders  the  analyst’s  ability  to  understand  the  military  environment.  ARL  is  in  the  process  of 
investigating  this  issue  by  redesigning  the  basic  algorithms  that  underlie  the  co-evolutionary  and 
genetic  approaches  and  by  generating  visualization  for  the  non-traditional  multi-agent,  volatile 
situation  such  as  anti-terrorist  campaigns  in  Afghanistan.  The  results  will  be  evaluated  with 
experienced  analysts  to  help  design  better  visualization  systems  and  to  pinpoint  problems  with 
the  basic  concept.  Tools  based  on  co-evolutionary  processes  show  particular  promise  for 
understanding  complexity  in  that  they  deliberately  mimic  the  chaos  of  battlefield  situations  by 
using  dynamic  algorithms  that  have  no  predetermined  solutions  and  are  particularly  useful  for 
discovering  unanticipated  results  (Hillis  &  Winkler,  2001). 

An  unfortunate  mistake  in  trying  to  evaluate  systems  that  are  being  designed  for  complex 
environments  is  the  belief  that  these  systems  must  be  subjected  to  realistic  validation  early  in 
their  development  process.  Evaluations  must  be  a  multi-step  process  from  early  modeling  with 
expert  feedback  to  controlled  experiments  to  field  validations.  Too  often  in  the  attempt  to  capture 
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combat  complexity,  the  temptation  to  skip  steps  leads  to  disastrous  field  experiences  because 
promising  technologies  are  introduced  prematurely  (Barnes  &  Beevis,  in  press).  An  example  of 
an  attempt  to  introduce  realism  and  control  early  in  the  design  process  is  an  ARL-supported 
project  at  the  University  of  Arizona.  The  researchers  are  trying  to  circumvent  these  problems  by 
developing  a  simple  simulation  environment  for  initial  evaluations  of  visualization  and  decision 
support  concepts.  The  purpose  is  to  combine  the  efficiencies  of  iterative  feedback  from  experts 
and  controls  of  laboratory  experiments  with  at  least  some  of  the  complexity  of  full-scale  simula¬ 
tions.  The  3-D  visualization  architecture  has  been  used  to  investigate  the  utility  of  employing 
various  predictive  and  assessment  aids  to  enhance  the  commander’s  ability  to  operate  in  noncon- 
ventional  situations  such  as  Kosovo,  Afghanistan,  and  Somalia  (Rozenblit,  Barnes,  Momen, 
Quijada,  &  Fichtl,  2000;  Ziegler,  Rozenblit,  Barnes,  &  Hudlicka,  2000).  This  architecture  allows 
the  experimental  manipulation  of  visualization  and  algorithmic  techniques  via  realistic  military 
vignettes  with  active  duty  officers  as  test  subjects.  Figure  5  portrays  examples  of  concepts  that 
were  demonstrated  to  groups  of  military  experts  to  assess  the  military  utility  of  various  3-D 
animation  and  configural  display  concepts.  Future  efforts  include  data  collection  during  realistic 
simulated  small-scale  contingency  operations. 


Figure  5.  Three-dimensional  visualizations  showing  possible  options  combined  with  predictive  software 
and  configural  displays  that  show  the  consequences  of  the  animated  vignettes  (Rozenblit  et  al., 
2000). 
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5.  Conclusions 


Visualization  is  a  problem-solving  stratagem  that  uses  internal  and  external  images  to  enhance 
the  commander’s  ability  to  understand  the  current  situation  and  to  envision  the  actions  necessary 
to  attain  combat  goals.  Viewing  battlefield  terrain  is  a  complicated  process  that  requires  2-D 
planar  views  for  the  understanding  of  tactical  and  spatial  relationships,  3-D  views  for  topograph¬ 
ical  understanding,  and  immersed  3-D  views  for  mission  rehearsal  and  detailed  understanding  of 
the  battlefield  “dirt.”  The  principal  research  issue  is  to  develop  principles  for  the  integration  (or 
separation)  of  various  views  so  that  battle  managers  can  perceive  tactical  and  topographical 
battle  conditions  as  a  seamless  narrative.  Military  symbols  are  products  of  tradition  rather  than 
human  engineering  efficiency.  Coding  techniques  such  as  color  and  creation  of  more  uniform 
symbol  sets  have  increased  the  processing  efficiency  of  modem  symbol  sets  without  solving  the 
basic  problems  of  clutter  and  poor  contrast.  New  display  concepts  based  on  research  findings 
related  to  pre-attentive  processing  and  emergent  perceptual  cues  may  circumvent  at  least  some  of 
the  problems  related  to  the  portrayal  of  the  complexity  of  modem  battlefields.  Also,  knowledge 
wall  technologies  are  being  developed  to  display  all  pertinent  information  (including  live  battle¬ 
field  imagery)  to  the  commander  as  a  mosaic  of  various  knowledge  sources.  The  problem 
remains  as  to  how  to  present  multiple  sources  of  information  with  techniques  that  aid  die  com¬ 
mander  in  visualizing  not  only  the  current  situation  but  also  in  predicting  the  most  effective 
responses  and  future  COAs.  The  most  important  human-related  design  issues  are 

•  Projecting  a  3-D  representation  onto  a  2-D  surface  results  in  space  distortions, 
ambiguous  locations  along  the  line  of  sight,  and  foreshortening  phenomena.  Visualization  cues 
such  as  mesh  and  line  enhancements  help  us  understand  3-D  views. 

•  Tactical  information  is  displayed  more  effectively  with  2-D  planar  formats,  whereas 
understanding  the  topography  is  best  visualized  with  3-D  formats. 

•  There  is  a  cognitive  cost  in  switching  between  dimensionality  and  slant  range  views. 
Software  utilities  that  allow  the  observer  to  morph  between  slant  range  and  dimensionality  views 
of  the  same  terrain  have  proved  effective  in  laboratory  experiments. 

•  In  the  future,  warfighters  may  be  able  to  visualize  their  military  situation  using  VE. 
However,  many  costs  are  currently  associated  with  VE,  which  must  be  overcome  before  the 
technology  becomes  used  widely:  motion  sickness,  perceptual  illusions,  time  lags,  and  observer 
disorientation. 

•  A  small  set  of  symbols  with  distinct  shapes  permits  observers  to  use  pre-attentive 
processing  mechanisms  to  maintain  overall  battle  awareness. 
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•  Military  symbols  in  general  and  Army  symbols  in  particular  are  only  minimally 
effective  in  portraying  complex  military  situations.  Because  many  of  these  symbol  sets  have 
been  used  for  generations,  they  are  unlikely  to  be  supplanted. 

•  Properly  designed  color  coding,  even  with  its  own  perceptual  costs,  can  be  used 
effectively  to  enhance  understanding  of  situation  displays  via  military  symbol  sets. 

•  New  visualization  techniques,  such  as  configural  displays,  proximity  functions,  and 
narrative  formats,  which  are  based  on  human  pattern  recognition  abilities,  are  promising 
techniques  to  display  complex  military  situations  without  clutter  or  information  overload. 

•  Knowledge  walls  using  modem  digital  technology  are  being  designed  for  real-time, 
multi-source,  interactive,  and  shared  command  views  of  future  battlefields. 

•  Expert  users  emphasize  that  future  systems  must  be  collaborative,  intuitive,  broad 
based,  have  “drill-down”  capabilities,  and  be  predicated  on  the  CCIRs. 

Prediction  is  an  important  but  controversial  component  of  the  visualization  process.  The  litera¬ 
ture  about  biases  suggests  that  die  use  of  heuristics  will  result  in  humans  who  are  visualizing 
only  a  limited  portion  of  the  underlying  problem  space,  which  will  lead  to  sub-optimal  decisions. 
The  literature  about  naturalistic  decision  making  implies  the  opposite,  mainly  that  heuristic 
processes  allow  the  expert  to  react  to  difficult  situations  by  rapidly  finding  a  satisfactory  solu¬ 
tion.  There  is  some  truth  in  both  assertions;  humans  are  prone  to  certain  biases  but  also  show  an 
amazing  ability  to  adapt  to  difficult  situations  that  fall  into  their  area  of  expertise.  The  crucial 
task  for  the  designer  is  to  find  visualization  cues  that  alert  observers  to  critical  decision  parame¬ 
ters  that  are  consonant  with  their  level  of  expertise.  The  literature  suggests  that  even  simple 
manipulations  of  displayed  information  (such  as  when  information  was  presented  in  the  decision 
process,  type  of  feedback,  and  how  uncertainty  was  presented)  can  have  positive  effects  on 
performance.  In  contrast,  cognitive  biases  such  as  complacency,  anchoring  effects,  automation 
paradox,  and  distrust  of  automated  systems  resulted  in  ineffective  military  decisions. 

The  general  conclusion  is  that  most  of  these  biases  could  be  overcome  by  a  combination  of 
training  and  improved  methods  of  visualization.  In  particular,  visualization  concepts  based  on 
storytelling  would  seem  to  be  a  creative  approach  for  both  training  and  operational  knowledge 
presentation.  However,  these  methods  must  be  developed  in  realistic  environments  and  evaluated 
in  actual  field  environments  to  be  effective.  Finally,  cognitive  technologies  that  are  being 
developed  to  help  us  better  understand  military  complexity  were  assessed.  Bayesian  techniques 
to  sort  masses  of  intelligence  indicators  into  possible  enemy  actions  and  genetic  and  co-evolu- 
tionary  algorithms  were  found  to  be  potentially  useful  (but  as  yet  unproven)  techniques  to  help 
an  analyst  visualize  possible  sources  of  complexity  for  conventional  and  contingency  operations. 
Methods  to  evaluate  these  technologies  were  also  reviewed: 

•  Measures  of  uncertainty  based  on  frequency  counts  offer  a  promising  method  for 
displaying  uncertainty. 
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•  Because  cognitive  biases  are  often  based  on  rational  expectations  of  real-world 
processes,  training  with  realistic  simulations  is  necessary  to  allow  the  operator  to  visualize  the 
outcome  of  these  processes. 

•  Results  from  the  NDM  literature  suggest  that  simulations,  role-playing  and  story¬ 
telling  exercises  should  improve  the  novice’s  ability  to  visualize  solutions  to  novel  situations. 

•  When  information  is  presented  in  the  decision  cycle,  the  type  of  feedback  and  type  of 
automation  errors  influence  the  operator’s  perceived  role  in  using  automated  systems. 

*  •  To  achieve  automation  synergy,  visualization  systems  must  be  designed  to  keep  the 

operator  actively  engaged  and  knowledgeable  about  the  expected  outcomes  of  the  automated 
decision  process. 

•  Tools  based  on  co-evolutionary  processes  show  particular  promise  for  visualizing 
complexity  in  that  they  deliberately  mimic  the  chaos  of  battlefield  situations  by  using  dynamic 
algorithms  that  have  no  predetermined  solutions  and  are  particularly  useful  for  discovering 
unanticipated  results. 

•  Evaluating  visualization  tools  is  a  multi-step  process  that  requires  early  modeling  and 
experimentation,  continual  user  involvement,  as  well  as  realistic  validation  exercises. 

We  are  just  beginning  to  understand  how  to  design  a  visualization  architecture  that  is  responsive 
to  the  human’s  natural  decision-making  style.  However,  certain  features  of  this  architecture  are 
evident  The  top-level  display  process  will  focus  on  situational  understanding.  The  visualization 
will  be  knowledge  rich  but  data  sparse,  focusing  on  the  decision  elements  that  allow  the  com¬ 
mander  to  follow  the  course  of  the  battle  narrative.  Beneath  the  display  surface,  multiple  battle 
parameters  will  be  continuously  updated,  and  triggers  will  surface  information  that  indicates 
possible  problems.  More  sophisticated  predictive  algorithms  and  collaborating  humans  will  be 
computing  in  the  background  and  forecasting  the  battle  process.  These  human  and  machine 
analysts  will  interrupt  only  if  objectives  are  threatened  or  unusual  trends  develop.  A  variety  of 
tools  that  address  specific  problems  and  planning  requirements  is  also  needed.  This  suggests  that 
the  visualization  architecture  will  be  collaborative,  with  each  component  having  private  and 
»  public  visualization  aids.  The  components  will  be  widely  disbursed  and  will  consist  of  battle 

staff,  off-site  expertise,  and  autonomous  intelligent  systems  networked  into  a  common  cognitive 
v  architecture.  An  important  design  issue  will  be  to  ensure  that  the  individual  components  act 

synergistically  to  create  a  unified  and  coherent  view  of  the  battiespace,  which  suggests  the 
importance  of  filters  and  knowledge  management  protocols  in  future  systems.  Principles  based 
on  the  use  of  narrative  formats  are  possible  cognitive  solutions  for  improving  coherency.  The 
visualization  architecture  will  be  analogous  to  human  consciousness.  The  visualizations 
themselves  will  be  the  top-level  component  that  imparts  meaning  and  understanding  to  the 
diverse  information  and  decision-making  components  serving  the  command  structure. 
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